International Journal of Medical Informatics
○ Elsevier BV
Preprints posted in the last 90 days, ranked by how well they match International Journal of Medical Informatics's content profile, based on 26 papers previously published here. The average preprint has a 0.04% match score for this journal, so anything above that is already an above-average fit.
Veramendi-Espinoza, L. E.; De la Cruz-Torralva, K.; Pezo-Pezo, A.; Vargas-Herrera, J. R.; Neyra Quijandria, J.; Martina, M.; Sullivan, K. E.; Huaman, M. A.; Knapke, J. M.
Show abstract
Background: Peru lacks an integrated information system for patients with Inborn Errors of Immunity (IEI). Although disease registries are essential tools for data management and health planning, their success depends on implementation science approaches that account for local contextual factors. This study reports Phase I of a three-phase mixed-methods implementation project to design and develop a national IEI registry. Methods: Phase I consisted of a phenomenological qualitative study exploring stakeholder perspectives. Semi-structured focus groups and in-depth interviews were conducted with 29 key stakeholders across four groups: policy-makers, clinical experts, end-users (immunologists, residents, allied health personnel), and patient organization representatives. Interviews followed a guide structured around four a priori domains (structure, navigation, feasibility, and perception of existing systems). Discussions were conducted in Spanish, audio-recorded, transcribed verbatim, and coded using ATLAS.ti. A hybrid thematic analysis combining deductive and inductive coding was performed. Data elements proposed for the registry were triangulated with qualitative findings. Results: Thirty-six initial codes were consolidated into 15 categories, which were further integrated into four overarching themes conceptualized as pathways toward intention to use: (1) Environment, where governance, regulatory backing, and sustainable financing were identified as key enablers, while limited interoperability emerged as a structural barrier; (2) Technical Dimension, emphasizing usability, alignment with clinical workflow, and a hierarchical data architecture (demographic, clinical, therapeutic); (3) Users, highlighting clinical leadership, protected time, digital readiness, and perceived usefulness as stronger motivators than financial incentives; and (4) Patients, underscoring data protection, transparency, trust, and advocacy as essential for legitimacy and sustainability. Conclusions: A national IEI registry in Peru is perceived as necessary and feasible if implemented with strong regulatory foundations, interoperable design, robust data security, and user-centered architecture. These findings informed the development of an initial functional prototype and the operational plan for Phase II, focused on usability evaluation.
Dojcsak, L.; Abegaz, T.; Islam, M.; Chandler, Y.; Maleku, A.; Doubeni, A.; Mohammed, B.; Langston, M. A.; Donneyong, M. M.
Show abstract
Health-related social needs (HRSNs), such as housing instability, food insecurity, and transportation challenges, are nonmedical factors associated with poorer health and well-being. Screening for unmet HRSNs is a critical step towards identifying at-risk patients, but manual screening is resource intensive and often incomplete. We utilized Electronic Health Records (EHR) data to develop machine learning models to identify unmet HRSNs using a limited set of non-modifiable sociodemographic features available in EHRs. We included 745,975 patients screened for at least one HRSN using data from community health centers that participated in the OCHIN practice-based research network between 2016 and 2022. Logistic regression, random forest (RF), eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM) algorithms were trained to predict unmet HRSNs. Model performance was evaluated using 10-fold cross-validation and area under the receiver operating characteristic curve (AUROC). For overall HRSN prediction, LightGBM (AUROC, 64.5%, 95%CI: 64.3, 64.7) performed slightly better than logistic regression (61.4%), RF (63.7%), and XGBoost (60.3%). Similar performances were observed predicting individual HRSNs. Model performances were modest; however, they establish a benchmark for predictive performance achievable using only routinely available demographic data and provide a foundation for incorporating additional clinical and area-level social determinants of health data.
Zhuang, H.; Zakama, A.; Heller, K.; Faulkner, S.; Gollub, B.; Young-Lin, N.; Chen, I. Y.; Asiedu, M.
Show abstract
In this work, we demonstrate the unprecedented value of NIH's "All of Us Research Program" (AoURP) dataset in studying maternal morbidity and building predictive machine learning (ML) models across heterogeneous populations in the United States. We developed robust and data-driven preprocessing pipelines to curate a longitudinal, multi-site, multimodal, and demographically diverse pregnancy dataset (20,253 subjects; 27,525 pregnancy episodes) from AoURP data, using electronic health records (EHR) (Conditions, Labs, Measurements) and survey responses (Social Determinant of Health (SDoH)), focusing on 7 crucial maternal health adverse outcomes. After characterizing data quality, missingness, and heterogeneity, we performed statistical correlation analysis to identify risk factors. We subsequently developed XGBoost and sequential LSTM models to predict the adverse outcomes, reaching state-of-the-art performance for multiple outcomes. We conducted model interpretability post-hoc analysis to understand success points and fairness analysis to evaluate implications for socio-economic disparities. Four practicing physicians reviewed the set of statistically significant and ML model identified features to assess their clinical validity and novelty. Most features identified through either statistical correlations or ML feature importance analysis aligned with known clinical risk factors. Several features were identified that the ML models used but that are not currently used in clinical practice and may merit further clinical investigation. Fairness analysis revealed certain associations with SDoH and age highlight areas that warrant continued monitoring. Overall, we demonstrate that meaningful populational level patterns can be extracted, and high-performing machine learning models can be trained on this longitudinal, diverse, multi-site dataset. Important risk features, particularly novel ones identified, if validated, could inform new strategies for maternal care or enable development and validation of outcome-specific, clinically deployable ML models.
Champeaux, S. A.; Booth, J.; Brown, A.; Sebire, N. J.; Drobnjak, I.; Bowyer, S.
Show abstract
Background: Machine learning models leveraging electronic health records (EHRs) can support earlier detection of sepsis in intensive care units (ICUs). However, their clinical utility depends on reproducibility across institutions and patient populations. Building on a published pipeline from the Children's Hospital of Philadelphia (CHOP), this study examines how a neonatal sepsis prediction framework performs and can be adapted to a range of intensive care environments, paediatric, cardiac, and neonatal, at Great Ormond Street Hospital (GOSH). Methods: We extracted de-identified ICU EHR data from GOSH and applied feature derivation, unit harmonisation, and temporal sampling to align with the CHOP dataset used by Masino et al. (2019). Seven classifiers were first evaluated using CHOP-trained weights to characterise cross-domain behaviour and then retrained on local data to assess recoverability and site-specific adaptation. Model discrimination was summarised by AUC and F1, and learning curves were used to explore sample efficiency and bias-variance dynamics. Results: Models achieved strong discrimination on the CHOP neonatal cohort but demonstrated reduced performance when transferred to the mixed GOSH ICU population, reflecting anticipated domain and population shift. Retraining on GOSH data restored discrimination (AUC range 0.69-0.86), with Gradient Boosting (AUC 0.86 vs AUC 0.87 at CHOP) and KNN (AUC 0.80 vs AUC 0.79 at CHOP) models performing comparably to their CHOP benchmarks. DeLong's test confirmed statistically significant gains across all classifiers (p < 0.001). Conclusion: ICU cohort and baseline demographic differences between CHOP and GOSH introduced domain shift that limited direct model transfer. Elements of the original preprocessing pipeline could not be reproduced, further constraining transportability. Yet, retraining on local data restored high discrimination, showing that the modelling framework remains robust when re-estimated in new settings. These results highlight local adaptation as a practical route to recover performance and support safe, generalisable deployment of clinical prediction models in mixed clinical environments.
Takeda, A.; Igata, H.; Mizuno, K.; Yano, Y.; Nagasu, H.; Ohashi, M.; Kashihara, N.; Kobayashi, H.
Show abstract
Predicting the long-term kidney function decline is critical for timely intervention but remains challenging. While the urinary protein-to-creatinine ratio (uPCR) is a potential surrogate endpoint, its short-term reduction's link to long-term nephroprotection requires investigation. This study aimed to develop a probabilistic neural network model to capture both the estimated glomerular filtration rate (eGFR) slope and its uncertainty based on baseline clinical characteristics. Using a retrospective dataset, we designed a neural network to output a predictive distribution (mean and standard deviation {sigma}) for the eGFR slope. SHAP (SHapley Additive exPlanations) was used for model interpretation, and a simulation study quantified the impact of uPCR reduction. In the validation set, the model achieved a Pearson's correlation coefficient of 0.56 and an RMSE of 2.81 ml/min/1.73m^2/year between predicted and actual slopes. SHAP analysis identified uPCR as the most potent predictor, with higher baseline levels associated with a more rapid eGFR decline. Furthermore, a simulated 62% uPCR reduction demonstrated a significant improvement in the predicted eGFR slope, an effect most pronounced in patients with high baseline uPCR. This proof-of-concept study reinforces the critical role of uPCR in predicting eGFR slope and suggests its reduction may contribute to long-term kidney function preservation, warranting validation in larger, diverse real-world datasets.
Kim, Y.; Lee, Y.; Jeong, J.
Show abstract
Public healthcare databases in South Korea have been distributed across disparate government agencies, requiring researchers to navigate multiple, separate institutional approval processes for data linkage. To address the systemic inefficiency, the Healthcare Big Data Linkage Platform (HCDL), jointly administered by NECA and KHIS, was established to integrate 13 databases from 10 public institutions through a Trusted Third Party (TTP)-based linkage methodology and a centralized one-stop review process. Of 311 projects submitted between 2022 and 2025, 190 (61.1%) were approved with annual applications increasing 2.4-fold over the study period. The average number of databases per project exceeded three, reflecting a surging demand for integrated clinical data. Nationwide healthcare data from HIRA and NHIS were the most frequently requested databases (93.2% and 82.1% of approved projects, respectively), and co-occurrence pattern analysis further confirmed that both formed the core of the research ecosystem in combination with vital status, lifestyle, and cancer diagnosis data. By consolidating multi-institutional review and enabling equitable data access, the HCDL has emerged as a core infrastructure for data-driven and precision medicine research in South Korea.
Naderalvojoud, B.; Sutjiadi, B. J.; Koul, A.; Curtin, C.; Gevaert, O.; Hernandez-Boussard, T.
Show abstract
Background Machine learning (ML) models are increasingly used to predict adverse outcomes after surgery. However, most rely on static patient characteristics (e.g., age, comorbidities) and overlook clinician-controlled treatment decisions that can be actively modified at the point of care. Discharge opioid prescribing is a key modifiable, clinician-controlled decision, yet optimizing prescribing choices across multiple adverse outcomes remains underexplored in predictive modeling. This study addresses that gap by introducing a novel ML framework that explicitly separates fixed patient risk factors from modifiable prescribing options to support personalized, risk-informed opioid prescribing decisions. Methods We developed the Hierarchical Clinical Fusion Transformer (HCF-Transformer), an ML model designed to estimate patient-specific risks across four postoperative outcomes: prolonged opioid use (POU), chronic pain (CP), 30-day readmission, and opioid-associated outcomes (OAO). The model constructs patient risk profiles from fixed, non-modifiable baseline factors, followed by a transformer layer. Clinician-controllable discharge opioid regimens are modeled as alternative intervention candidates and fused with the fixed risk representation through a clinical fusion mechanism, enabling assessment and ranking based on predicted risks. A Total Relative Risk (TRR) metric, calibrated to each outcome prediction threshold, guides the recommendation process. We evaluated the model in diabetic surgical patients, a common high-risk population. Results The study included 157,853 unique diabetic surgical patients, with outcome prevalences ranging from 47.2% (POU) to 1.8% (OAO). The HCF-Transformer achieved the highest AUROCs, 0.798 for POU, 0.712 for 30-day readmission, 0.808 for CP, and 0.922 for OAO, outperforming Random Forest, FT-Transformer, and ResNet-based models. Compared to these baselines, HCF-Transformer generated more stable and discriminative risk estimates and demonstrated significant variation in TRR scores across discharge opioid options (ANOVA p < .01, eta-squared > .01). This enabled consistent identification of lower-risk regimens tailored to patient-specific profiles. Conclusions The HCF-Transformer introduces a novel hierarchical fusion approach to optimize opioid prescribing by integrating static patient risk profiles with modifiable discharge options. Using transformer-based modeling and a quantifiable TRR metric, the model delivers personalized, risk-aware recommendations. This approach enables data-driven opioid prescribing tailored to individual risk and has the potential to improve postoperative outcomes in high-risk populations. Our findings demonstrate that integrating modifiable factors with structured risk profiles through a transformer-based fusion architecture can enhance decision-support systems, paving the way for more actionable and personalized AI in healthcare.
Okundaye, D. O.; Isiekwene, C. C.
Show abstract
Acute kidney injury (AKI) is a frequent complication within intensive care units, with its sudden onset often missed. This is especially important because a timely window for intervention is required as delayed detection leads to progressively worse outcomes. Existing machine learning and deep learning models have contributed to closing this gap, but their complexity, requiring hundreds to thousands of features, and lack of generalisation pose a limitation that prevents them from being integrated into clinical workflows across different electronic health-record ecosystems. This study presents a 37-feature XGBoost model trained on the MIMIC-IV dataset with 5.4% positive cases, with hyperparameters optimised via Optuna and probabilities calibrated using isotonic regression, designed for transportability across clinical settings. Validation was conducted internally using a temporal patient-level split simulating prospective deployment, training on 2008-2016 data and testing on 2017-2022 data"External validation was performed on the eICU Collaborative Research Database, a multi-centre dataset spanning 208 US hospitals, using the trained model without retraining. SHAP TreeExplainer was used to provide feature-level explainability for individual predictions. Internal testing yielded an AUROC score of 0.794 for predicting AKI onset within a 12-24 hour window. External validation produced a 0.750 AUROC without retraining. Equitable discrimination was observed across gender, age, chronic kidney disease presence, race, and AKI stages on both datasets, with a 95% internal CI of 0.789-0.799 confirming the model's estimate stability. These results suggest that clinically useful prediction systems are achievable with substantially fewer features than current models require.
Schack, M.; Rathert, H.; Boehnke, J.; Ruebsamen, N.; Bode, L.; Karch, A.; Almekkawi, M. K.; Marschollek, M.; Beerbaum, P.; Wulff, A.; Jack, T.
Show abstract
Background: Sepsis is a life-threatening condition triggered by infection and associated with dysregulated immune response of the patient followed often by multiorgan dysfunction or failure. In the clinical evolution of sepsis towards organ dysfunction, early initiation of a suited therapy significantly increases patient outcomes and reduces mortality rates. Since electronic health records provide data in a machine-readable format, this process could be supported by computerized systems. Methods: We developed an interoperable, time-sensitive CDSS that able to detect systemic inflammation and the different classifications of sepsis (bacterial/viral, suspected/proven, on admission/PICU acquired) in pediatric patients based on the analysis of routine clinical data. This application is provided as part of this publication as an open demonstrator (web application), and the usability and accuracy of the CDSS is shown by a retrospective creation of sepsis outcome labels for a routine data set of 4,655 pediatric patients. As a reference standard, the patients were manually assessed by blinded clinical experts. Results: In comparison with the reference standard, the CDSS achieved sensitivity of 96.9% (95% CI: 80.9-99.6%) and specificity of 99.1% (95% CI: 95.1-99.8%). In the context of a sepsis outcome labeling for 4,655 patients, the CDSS detected 4,342 episodes of inflammation of which 1,723 were classified as sepsis. Conclusions: We demonstrated that our routine-data based CDSS is able to perform a complex sepsis detection process with high diagnostic accuracy. Such CDSS with the ability to differentiate between SIRS, sepsis on admission, suspected and proven sepsis can prospectively support clinical management, monitoring and quality management.
Soejima, A.; Kitano, F.; Ichikawa, D.; Shibagaki, Y.; Noda, R.
Show abstract
Background: Whether benchmark performance reflects robust clinical reasoning rather than surface-level pattern recognition remains uncertain. We evaluated the robustness of state-of-the-art large language models (LLMs) on nephrology board renewal questions using "None of the other answers" (NOTA) substitution. Methods: From 210 Japanese Society of Nephrology board renewal questions (2014-2023), two nephrologists independently reviewed all items. Questions in which NOTA became the sole correct answer after replacement were included, yielding 145 validated questions. GPT-5, GPT-4o, Gemini 2.5 Pro, and Gemini 2.0 Flash were evaluated via application programming interfaces under default settings. The primary endpoint was accuracy, and paired differences were assessed using the exact two-sided McNemar test. Results: Accuracy was significantly lower after NOTA substitution for all models: GPT-4o, 66.21% to 19.31% (drop, 46.90 percentage points [pp]); GPT-5, 87.59% to 73.10% (14.48 pp); Gemini 2.0 Flash, 58.62% to 31.03% (27.59 pp); and Gemini 2.5 Pro, 86.90% to 55.86% (31.03 pp); all P < .001. GPT-5 showed the smallest decline and the highest accuracy in both versions. Conclusions: All evaluated LLMs showed a significant robustness gap after NOTA replacement. Newer models may be more robust, but multiple-choice accuracy remains an incomplete measure of clinical reasoning robustness.
Badhon, S. M. S. I.; Adibuzzaman, M.; Mosa, A. S. M.; Bozdag, S.; Cleveland, A. D.; Ding, J.; Hossain, K. S. M. T.
Show abstract
Objective: Acute kidney injury (AKI) affects a large proportion of patients in the intensive care unit (ICU) and is a major contributor to morbidity, mortality, and cost. Although electronic health records (EHRs) capture rich longitudinal data, many predictive models fail to detect AKI early enough for effective intervention. Non-temporal methods such as logistic regression and XGBoost treat patient history as aggregated risk factors, discarding the temporal evolution of clinical state. A recent trend is to employ temporal models, such as recurrent neural networks, to capture sequential patterns, but these models struggle with irregular sampling and limited long- range contextual awareness. To address the challenge, we propose RenalTransLSTM, a hybrid temporal deep learning framework for early, multi-horizon AKI prediction and identification of modifiable risk factors. Methods: RenalTransLSTM integrates Long Short-Term Memory (LSTM) networks with Transformer encoders to model both local temporal dynamics and global contextual depen- dencies in ICU time-series data. Using 48-hour patient histories from MIMIC-IV (61,735 admissions), the model predicts AKI at 6-, 12-, and 24-hour lead times. We benchmark the model against SVM, XGBoost, LSTM, TG-LSTM, and a Transformer, and apply Integrated Gradients and counterfactual analysis to identify modifiable risk factors. Results: RenalTransLSTM outperforms all baselines across most horizons and metrics, achiev- ing AUROC above 0.90 and F1-scores reaching 0.85 while maintaining balanced precision and recall on imbalanced data. Ablation studies confirm that combining LSTM and Transformer components improved robustness and predictive performance. Counterfactual analysis identifies clinically meaningful, modifiable risk factors associated with AKI progression. Conclusion: RenalTransLSTM offers an effective, interpretable framework for early AKI prediction in the ICU, supporting proactive intervention and clinical decision support.
Patchigolla, V.; Jhand, A. S.; Lee, H. J.; Benjamins, L. J.
Show abstract
Evidence-based medicine (EBM) concepts are difficult for medical students to grasp. We developed a Python-Streamlit web application providing interactive visualizations to enhance EBM education. Preliminary use with first year medical students demonstrated high engagement and improved conceptual understanding, supporting the feasibility of integrating interactive, web-based tools into EBM curricula.
Sanjaya, J.; Pathak, S.; Si, Y.; Haghi, M.; Kudrot, N.; Placencia, G.; Alaei, K.; Pishgar, M.
Show abstract
Background: Critically ill patients with cancer and sepsis have high in-hospital mortality, but externally validated prediction models are limited. Objective: To develop and externally validate an interpretable machine learning framework using first-day intensive care data. Methods: We used MIMIC-IV version 3.1 for development and internal validation and eICU-CRD for external validation. Eligible adults had cancer, an intensive care unit stay of at least 24 hours, and met a prespecified operational sepsis definition. The prediction landmark was 24 hours after admission. The MIMIC-IV cohort included 3,729 stays (training, n = 2, 983; internal validation, n = 746). Same-admission diagnosis-derived variables were excluded, and 345 predictors were retained. Fourteen predictive models and a dummy baseline were evaluated. Frozen pipelines and training-derived thresholds were applied to eICU-CRD without refitting or recalibration. Results: Gradient boosting was selected as the primary model and achieved an internal AUROC of 0.8480 (95% CI, 0.8173-0.8755), AUPRC of 0.6984, and Brier score of 0.1346. Important predictors included Glasgow Coma Scale components, temperature, lactate dehydrogenase, age, respiratory rate, oxygen saturation, blood urea nitrogen, and serum lactate. In eICU-CRD (n = 611), gradient boosting achieved an AUROC of 0.7483 (95% CI, 0.7026-0.7919), AUPRC of 0.6243, and Brier score of 0.1709. Random forest had the highest external AUROC in secondary comparisons (0.7731). Conclusions: First-day data supported useful internal discrimination, but performance declined under locked external validation. Multicenter validation, recalibration, threshold assessment, and prospective evaluation are required before clinical implementation.
Cousin, A.; Legrand, V.; Devillier, R.; Karam, M.; Forcade, E.; Jubert, C.; Villate, A.; Eloit, M.; Gyan, E.; Chevalier, P.; Labussiere-Wallet, H.; Castilla-Llorente, C.; Maertens, J.; Ceballos, P.; Rubio, M.-T.; Bruno, B.; Chalandon, Y.; Poire, X.; Mear, J.-B.; Gandemer, V.; Levy, J.; Malard, F.; Lewalle, P.; Paillard, C.; Loschi, M.; Dalle, J.-H.; Charbonnier, A.; Daguindau, E.; Bay, J.-O.; Prata De Lima, P.; Maillard, N.; Suarez, F.; Benakli, M.; Bazarbachi, A.; Thalhammer, J.; Nguyen, S.; Raus, N.; Huynh, A.; Michonneau, D.; Vallet, N.
Show abstract
Despite longitudinal and multidimensional collected data within registries, their routine exploitation for value-based care and outcome transparency remains limited by analytical complexity and heterogeneous expertise across centers. To address this gap, we developed an open-source and free web-based software which allows registry-based data analysis operational for evaluation of practices and quality system management applied to allogeneic hematopoietic cell transplant registry. It was built with Python and Dash framework to treat user formatted data. AlloGraph produces epidemiological summaries, survival analyses, and quality management indicators. Privacy protection is ensured by a Transport Layer Security protocol to a secure server where processing occurs in-memory, without data saving. AlloGraph was evaluated positively by 30 practitioners in 24 transplant centers, of whom 89% anticipated that AlloGraph would change their monitoring practice. AlloGraph represents a privacy-preserving and user-centered platform simplifying registry analysis for activity monitoring. This scalable model could be adapted to exploit real-world health databases.
Santos, R. d. P.; Tinoco Patricio, A. d. O.; Gama, P. H.; Freitas, L. M. D.; Ribeiro, K. R.
Show abstract
Objective: To construct and evaluate, in an exploratory manner, a pathophysiologic rationale link- ing biological pathways derived from the peripheral transcriptome in ischemic stroke (IS) to nursing diagnoses in the NANDA-I 2024-2026 taxonomy, while emphasizing that this association is not di- rect, deterministic, or automatically inferable from textual similarity with large language models (LLMs). Methods: A computational study was conducted using public secondary data from the Gene Ex- pression Omnibus series GSE16561, which includes 63 peripheral blood samples: 39 from indi- viduals with IS and 24 from healthy controls. The pipeline integrated transcriptomic analysis and functional enrichment, semantic mapping through ClinicalBERT embeddings, and mechanistic and clinical-conceptual judgment using Claude Sonnet 4.6 as a judge. The judgment stage was treated as the central interpretive layer, designed to mediate the transcriptome, pathophysiology, functional manifestation, and NANDA-I diagnosis. Results: The analysis identified a bimodal transcriptomic pattern, with activation of pathways re- lated to innate immunity and suppression of pathways related to adaptive immunity. Semantic map- ping generated 158 pathway-diagnosis pairs. The Spearman correlation between cosine similarity and the mechanistic score was negative and statistically significant (rho = -0.243; p = 2.09e-03), but weak in magnitude. This effect size indicates that semantic similarity explained less than 6% of the variance in mechanistic plausibility, reinforcing the insufficiency of embeddings as a stand- alone criterion. Of the 158 pairs, 14 were classified as high concordance, 8 as moderate, and 136 as divergent. Conclusion: The main value of this study lies in demonstrating that translating biological pathways into nursing diagnoses requires pathophysiologic, functional, and clinical-conceptual mediation. The prioritized pairs represent mechanistically plausible hypotheses for future research, without implying causality, direct clinical confirmation, or immediate care recommendations.
Steitz, B. D.; Ogunsan, O. O.; Ancker, J. S.; Carlson, B. R.; Gaynor, L. S.; Higashi, R. T.; Morrow, E. L.; Reese, T. J.; Romano, R. R.; Stern, S.; Turer, R. W.; Rosenbloom, S. T.; Wright, A.
Show abstract
Objectives: Characterizing patient portal message content at scale can help target efforts to manage administrative work. We developed and validated a large language model (LLM) pipeline for multi-label classification of messages using an expert-derived topic taxonomy, then characterized topic distribution across a two-year corpus. Materials and Methods: We studied all medical advice request messages sent to ambulatory clinicians at an academic medical center from 2024-2025. We convened an expert panel that derived an 11-category taxonomy through a modified Delphi process. Two annotators labeled 750 randomly selected messages (Cohen kappa 0.80), holding out 500 for evaluation. The pipeline used GPT-4o-mini in a zero-shot prompt. On the held-out set, we measured micro- and macro-averaged precision, recall, and F1, and label stability across runs. We then characterized topic distribution and co-occurrence across the corpus. Results: The pipeline achieved micro- and macro-averaged F1 of 0.89 and 0.86. Labels were identical across runs for 93.6% of messages. Across 2.4 million messages, content concentrated on a few topics. The two most common topics, Problems & Management and Medications & Prescriptions, were present in 67.9% of messages, and the four most common in 93.9%. 51.7% of messages addressed multiple topics. Discussion and Conclusion: The pipeline classified patient message topics accurately and stably across millions of messages. Message content was concentrated within a small number of topics, highlighting opportunities for targeted interventions and enabling more efficient triage, routing, and patient-facing support.
Hui, J.; Xia, M.; Wilson, J.; Hill, E. D.; Scheer, A.; Franz, L.; Engelhard, M. M.; Goldstein, B. A.
Show abstract
The performance of an EHR-based deep learning model trained on a small sample can be improved if more data is collected. Instead of collecting more data, the model can be trained on additional data from an analogous external source. However, this risks the model learning patterns in the external data that do not generalize to the target sample. Furthermore, data use agreements often prohibit combining datasets with medical records of different sources. We consider utilizing pre-existing methods in continual learning, namely the elastic weight consolidation (EWC) loss function and variational continual learning (VCL), both of which are regularization-based methods that we use to borrow external data and incorporate parameters from a model on external data into local model training. To investigate the utility of this modeling framework, we consider two binary classification tasks: (1) predicting which children will be diagnosed with autism spectrum disorder (ASD) from medical claims up to 18 months, and (2) predicting which patients with end-stage renal disease (ESRD) will be re-hospitalized within 30 days. Target datasets were derived from Duke University's EHR warehouse, and external datasets were sourced from either NC Medicaid claims for the ASD prediction task, or the United States Renal Data System (USRDS) for the rehospitalization prediction task. For both of these tasks, borrowing models - using either the EWC loss function or VCL - performed similarly to that of a model trained only on the full external data, when the sample size of target data used to train the model was small. That is, while a model that does not borrow using our methods performed poorly in low data regimes, the borrowing model instead matched the performance of a model trained on external data even when sample size of target data was small. In addition, an analysis of model predictions showed that models with small samples are better calibrated and more functionally similar to a model trained only on external data when the sample size is small.
Rony, A. R.; Nahin, K. S. A.; Islam, T.; Asha, A. S.; Hossen, A.
Show abstract
Caesarean section in Bangladesh reached 51.8% of deliveries in 2025, and elective caesarean, meaning caesarean before labour began, reached 31.6%. Risk models built on national household surveys are increasingly proposed for pointing audit toward places where scheduled surgery is outrunning clinical need, but they are usually validated in ways that flatter them. Using the 2025 Bangladesh Multiple Indicator Cluster Survey, we developed four models on 9,538 women (logistic regression, elastic net, random forest, gradient boosting) and ran the same procedure under three validation designs: random five-fold cross-validation; five-fold cross-validation grouped by sampling cluster; and leave-one-division-out cross-validation. We also tested transfer between the 2019 and 2025 rounds and audited subgroup calibration. No model improved on logistic regression by a margin worth acting on: the area under the receiver operating characteristic curve ranged from 0.724 to 0.736 under cluster-grouped validation, a spread of 0.012. Validation design mattered far more than the algorithm. Grouping folds by sampling cluster changed discrimination by at most 0.0004, this survey contributing a median of 3 eligible women per enumeration area. Withholding a whole division cost 0.044 to 0.060, more than 100 times as much, and still cost 0.033 to 0.056 after the strongest predictor, an outcome-derived district rate, was removed from every model. A model fitted to 2019 data lost 0.083 when applied to 2025, and the two rounds agreed only moderately on which predictors mattered (Spearman rank correlation 0.61). Calibration held in every wealth quintile, both residence categories and seven of eight divisions; Sylhet was the exception. Elective caesarean is predictable from routine survey items, but that predictability is local. Cross-validation, including cluster-aware cross-validation, does not measure what a model would do in a district it has never seen; a geographic holdout is the cheapest design that does.
Viswanadham, R. V. N.; Jones, S. A.; Owens, K.; Richardson, S. I.
Show abstract
BACKGROUND Clinical decision support (CDS) systems can improve care quality, but their implications for equity remain uncertain. We examined whether provider response to CDS alerts differed by patient race and sex in primary care, and whether differences in alert exposure helped explain any observed variation. METHODS AND PRINCIPAL FINDINGS We conducted a retrospective study using EHR data from a New York City academic health system, focusing on alert-based CDS during outpatient primary care. Logistic regression was used to estimate the likelihood of alert engagement by patient race and sex, while adjusting for encounter and provider factors. We used a generalized structural equation model to assess mediation by alert type, decomposing direct and indirect effects of demographics on response. Direct effects suggest that providers may respond differently to alerts based on patient identity, consistent with interpersonal bias, in which implicit or explicit attitudes shape clinical behavior, and on the context of the visit. Indirect effects highlight disparities in how alerts are assigned across groups, indicating that algorithmic or systemic bias may be embedded within the technology itself. Estimated mediated pathways suggest that even when providers respond uniformly to alerts, unequal exposure can still produce inequitable outcomes. DISCUSSION The findings highlight that the type of CDS triggered plays a significant role in differential CDS responses, with provider- and patient-related factors evident in these differences. These findings underscore the need to evaluate not only provider behavior but also the logic and distribution of CDS tools themselves, as both can contribute to disparities in care delivery. Further research should also focus on looking for the potential health impact of the differential response.
Dhaubhadel, S.; Cohn, J. D.; Bhattacharya, T.; Ribeiro, R. M.; Ganguly, K.; Hengartner, N. W.; Tate, J. P.; Costa, L.; Ho, Y.-L.; Cho, K.; Costa, L.; Beckham, J. C.; Kimbrel, N. A.; Justice, A. C.; McMahon, B. H.
Show abstract
We present a data-driven framework to predict 15-year all-cause mortality using outpatient administrative records for 2.3 million Veterans in the largest integrated U.S. healthcare system. Rather than relying on predefined clinical phenotypes, we used the 1,000 most common outpatient medical codes from each of three data types/modalities: ICD-9 (Dx), Current Procedural Terminology (CPT), and prescription drugs (Rx), encoded as binary features. Using these features, we trained three machine learning (ML) algorithms (logistic regression with lasso, random forest, and a 3-layered feed-forward neural network) to predict 15-year mortality risk. The features were also mapped to variables for the widely used Charlson Comorbidity Index (CCI), Elixhauser, and Veterans Aging Cohort Study (VACS) indices, refitted for 15-year mortality prediction, for baseline comparison. All our models significantly outperformed the widely used CCI, Elixhauser, and VACS indices, with C-statistics ranging from 0.82 to 0.84 versus 0.739 to 0.804 for the baselines. Relative improvements in C-statistics of our approach over the baselines were consistent across different subgroups (age groups of <65 years, those 65+years, Blacks, Hispanics, etc.) Our approach enabled the identification of high-impact predictors with clinical grounding , without requiring hand-curated phenotypes. Cardiovascular diseases and mental health diagnoses/treatments emerged as leading long-term mortality indicators. Using unsupervised ML techniques including PCA and K-means clustering, we associated interpretable patterns and complex interactions between diagnoses and treatments, highlighting comorbidities, disease trajectories, and healthcare utilization patterns. The ability to achieve the predictive performance and algorithmically detect such relationships purely from outpatient data supports the scalability and broad applicability of our framework. This framework not only improves mortality risk stratification over existing clinical indices, but also enables better understanding of how medical codes, regardless of category, interact to predict long-term outcomes.